Spurious "Extracting tar content of undefined failed" #7212
Comments
|
Weird: Edit: Relevant commit: b1f21a8 |
|
The yarn/src/fetchers/tarball-fetcher.js Lines 244 to 246 in 4ad8816 Passing It does nothing to actually help the error though. |
|
I ran Full log: verbose.txt.gz |
|
related to #6312
|
|
This is on 1.15.2. |
Yes, my bad. "yarn install --network-concurrency 1" is working for me, a temporary, slow workaround. |
|
I used the 1.7 version before today, it never happened, and today I upgraded to the latest version (1.15.2), it will happen: Extracting tar content of undefined failed, the file appears to be corrupt |
|
I was seeing this fairly consistently on 1.16.0, and was finally able to track this down. Our setup is as follows:
This was causing an extra instance of yarn to run that script, which then tried to fetch packages needed by this lib outside of the expected order. Since the lib in question is mostly dead, and barely needed in our case I was able to resolve the problem by removing the prepare script. For everyone else having these problems I recommend removing any git packages you have referenced, and failing that to grep all the package.json files in your project for these reserved scripts since they may be the cause. This is almost certainly the case if you see this sort of duplication: Hope this helps root cause this in a more reliable fashion. |
|
@arcanis You mentioned in #6312 that it was closed because there was no further details. I believe my post above this one may explain what's happening here. It look like (as an educated guess) that the instance of yarn being called to service the special scripts like |
* try thing * Delete yarn.lock * bump react-multiversal to avoid yarn bug yarnpkg/yarn#7212 * back trick
THANK YOU! I had recently added .yarnrc with I started by piping yarn into a file e.g. luckily, I decided to check out /Users/xxxxx/Library/Caches/Yarn/v4/.tmp/e237f3006359452a9561519898e180fb.5025ebf2427d709adfb10ee07f7dc36517d03694.prepare/.yarnrc and that folder was still around - checked out the package.json and realized that it was a library that I had forked and included as my dependancy recently. I'm guessing that the prepare script isn't called when the package is included "naturally" through npm, but it is when it is included as a github repo. For the record, the package I forked was react-beautiful-dnd. Once I updated my fork to remove the prepare script, the phantom pipeline issues stopped appearing. THANK YOU! |
|
try this way in your or in your workspace: |
#6312 suggested adding '--network-concurrency 1' which ** seems ** to work. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@esutton |
|
I can confirm the issue seems to be related to a git dependency. But a workspace is not needed in my case to repro the problem. Also, in my case it happens on a Docker node:12.7-alpine (I tried even with 11.x) image, but on on a a standard |
|
I think this is a dupe of #2629 |
|
It looks different to me 🤔 |
|
The |
|
Update found a workaround that does work.... a git submodule and then have the location of the package be the local folder. Not ideal but it works. Think we're having this problem too details here #2629 (comment) seems related to having a dependency hosted on github.com |
We're experiencing what looks like yarnpkg/yarn#7212 when we try to install the bookreader repo through yarn. It says the prepare step on a git:// referenced dependency (which is what we use) might be causing this error (although this error doesn't happen at all when run on @jbuckner's machine with that exact setup :/). Trying the `preinstall` hook to see if that fixes things.
Prepare was causing us to get this issue: yarnpkg/yarn#7212 . preinstall is run before install but not before a publish; since we don't publish this package on npm anyways, this is fine.
Prepare was causing us to get this issue: yarnpkg/yarn#7212 . preinstall is run before install but not before a publish; since we don't publish this package on npm anyways, this is fine.
Prepare is causing random failures when installing dependencies via yarn on ZEIT. Use preinstall instead. yarnpkg/yarn#7212 (comment)
* run validation before yarn license * attempt to fix CI errors yarnpkg/yarn#7212
|
If anyone is experiencing this still, as we were on bitbucket. We have two private repos that are imported into our main one, each of these repos have a prepare script that runs husky install. The prepare script has been changed to the below And the script ci-skip-husky.js A .yarnrc file with the current options set child-concurrency being the main one. This has fixed our issues on node 16.15.0, yarn 1.22.18 |
Note: these options were removed in yarn v2. |
Original issue is using Yarn 1 not 2, still relevant. |
* run validation before yarn license * attempt to fix CI errors yarnpkg/yarn#7212
|
same problem appearing on a very fast fiber connection. why should be a problem of network concurrency? |
|
This isn't caused by network issues. It's because multiple dependency trees overlap and Yarn doesn't do a great job of preventing a shared dependency with a build step from being built concurrently. It's a race condition that |
|
I don't think the debugging of the issue is the problem, its simply that yarn 1 is out of support. Seems there are 3 options:
|
|
We are using yarn 3, Same problem |
|
in the pipeline |
* Update lazy-log package * Refactor stream logs API * Introducing ContainerLogsView component * Support logging for pyfunc image builder and batch job * Fix batch job's image builder log. Support prefixing log with pod & container name * Add batch job executor log * Dockerfile: Add git so we Yarn installation can succeed * Use node:14 as node-builder base image * Colorized the pod + container in log * Use actions/setup-node@v2 and node v14 * Update react-lazylog package to use gojekfarm to sovle yarn install issue https://github.com/yarnpkg/yarn/issues/7212\#issuecomment-493720324 * We still need react-lazylog's prepare script. so we revert to roman's react-lazylog version and use yarn install --network-concurrency 1 * Refactor stackdriver log * Fix API's unit test first * Add more test to cluster and log_service * Fix UI wording * Update swagger * Make sure color lib turned on * Add build-essential and etc isntallation on Mlflows' Dockerfile * Update API test * Use gojekfarm's react-lazylog fork * Update how to close channel; getContainerLogs async * Use request.Context() for termintation * Fix API test * Modularize pprof routes into a spearate function * Address aria's review * Use unbuffered channel for sending log line * Periodically update component list and address reviews * Simplify component refresh & log api context cancellation
|
is there update for this? |
Removing the |
there is a race condition in the prepare script that makes yarn install fail randomy when fetching packages from clm-api cf yarnpkg/yarn#7212
there is a race condition in the prepare script that makes yarn install fail randomy when fetching packages from clm-api cf yarnpkg/yarn#7212
|
For us, Like @Dan-Wood, we use many private packages installed directly from Intermittent errors are: And |
|
I remember when Yarn was the new hotness. If you were still using Npm, everything took too long. Yarn was the only sane choice. Then this happened. I dealt with it for a while with manual build scripts that manually installed dependencies from git sources in a temporary directory first - this seemed to reliably fix the problem but always felt kludgy. It was a bummer that Yarn has refused to fix the problem, or really even acknowledge that there is one, but at least I had a workaround and was happy enough to keep using Yarn. Then Yarn 2 came crashing into the party with it's own headaches. Notably intentionally breaking compatibility with many Npm lifecycle scripts. As far as I can tell, there is now no way to have a "prepare" process that works consistently between Npm and Yarn - there is no way, that I'm aware of, to publish TypeScript sources such that when they're installed via git dependency, they consistently compile into the final desired JavaScript in both Yarn and Npm (without extra unreliable monkey patches in the build scripts). We've had reliable repros here for years. How is this still a thing? Is there a reason to keep using Yarn? |
You're two major versions behind, three if we count the current RCs for the next major. We know the 1.x has a couple of rare but annoying bugs that are unfortunately quite hard to safely address (by safely, I mean without introducing other subtle bugs). This is in large part why, almost four years ago now, we decided to publish new majors that tackle this problem. While using Yarn 1.x is still possible, and if it works for you then you don't have to migrate just for the sake of it, it's nonetheless advised to upgrade if you're hitting frustrating issues like this one. As for problems with modern releases, I encourage you to open issues on their dedicated tracker. I don't however remember hearing about problems similar to the ones you mention (the In any case, I'll lock this thread as our team doesn't plan to merge bugfixes here. Understand that any change we make here has the potential to break many applications, so for the better or the worst we decided a long time ago to simply freeze the old 1.x codebase (only exceptions being potential security fixes, and migration-related improvements). |
and others
Followup on #6312. Potentially related issues: #7087, #6755, #6312
Details
yarn --version: 1.15.2rm ~/.cache/yarn ~/.npm ./node_modules -rf && yarn install --production --frozen-lockfilenode:latestdocker image.I can still reproduce the issue on the latest yarn version. I do have a git dependency (
"@magicfind/wf-build-engine": "https://gitlab.com/magicfind/warframe/wf-build-engine.git") and I'm fairly certain it's related to that, because I've been messing with it, though I don't know what exactly caused this to start happening.I've been hitting #5235 and trying to work around it, and while doing that started hitting #6312.
Locally I see three different behaviours:
This points to a race condition of some type?
Example outputs
All outputs below run the exact same command line, removing all the cache I can find. I've not found a reliable way to reproduce the issue 100% of the time.
The text was updated successfully, but these errors were encountered: